The present invention relates to an image coding apparatus which uses motion compensation inter-frame prediction and its motion vector detection method and, more particularly, to an image coding apparatus and its motion vector detection method according to a structure using an image coding LSI.
As a method for realizing the motion compensation inter-frame prediction, there is a method of reducing temporal redundancy by using a motion vector as information which indicates from which position in an image one before a present image, a part of the present image is moved. As a method of extracting this motion vector, there is a block matching method.
FIG. 8 is a diagram for explaining the principle of the block matching method. According to this method, a frame image to be coded as a target of coding is compared with a search range frame image in which a motion vector is searched. Then, a best match block 902 which is the most similar to a coding target block 906 in a frame to be coded 905, i.e., a block having an evaluation value which has the highest correlation is extracted from the search range in a search range frame 903, thereby detecting a motion vector 901. In this method, the wider the search range is, the more easily the similar block is detected.
As a prior art image coding apparatus, there is an apparatus which internally contains all coding circuits including a motion search unit for detecting a motion vector, thereby to reduce the number of components.
For example, Japanese Published Patent Application No.10-108199 discloses an apparatus for executing inter-frame image prediction coding. This apparatus comprises an image input unit, a motion search unit, a pixel value arithmetic unit, a variable length coding unit, a code output unit, a frame memory, a frame memory control unit connected to the frame memory, a control processor, and a host interface. The frame memory control unit decides time-shared data transfer in each prescribed processing unit period by the same scheduling method.
To be specific, as shown in FIGS. 6 and 7 of Japanese Published Patent Application No.10-108199, the frame memory control unit has a frame index register internally. By controlling this frame index register with the control processor, the control unit changes an area for writing image data which is input by the image input unit or an area for writing a predicted image which is obtained by the pixel value arithmetic unit. This Patent Application No.10-108199 clearly describes that an image coding apparatus comprising fewer components, for example, an image coding apparatus comprising one outboard memory can be realized with this construction.
A bandwidth required for transferring data of a coding target of a common SD (standard density) image (720 pixelsxc3x97480 linesxc3x9730 frames/sec) of NTSC is given by Expression 1. In addition, a bandwidth required for transferring data of a search range in the case of a search range area of 48 pixelsxc3x9748 lines (xc2x116 pixels in the horizontal direction and xc2x116 lines in the vertical direction) is given by Expression 2.
((720 pixelsxc3x97480 lines+360 
pixelsxc3x97240 linesxc3x972)xc3x97
30 frames/secxc3x978 bits/pixel)xc3x97
2=0.25 Gbpsxe2x80x83xe2x80x83(Expression 1)
((720 pixelsxc3x97480 lines+
360 pixelsxc3x97240 linesxc3x972)xc3x97
30 frames/secxc3x978 bits/pixel)+
((720 pixelsxc3x97480 lines)xc3x97
(48 pixelsxc3x9748 lines)/(16 
pixelsxc3x9716 lines)xc3x9730 frames/secxc3x97
8 bits/pixel)xc3x972+(360 pixelsxc3x97240 linesxc3x972)xc3x9730 
frames/secxc3x978 bits/pixel)xc3x97
2=1.7 Gbpsxe2x80x83xe2x80x83(Expression 2)
Expression 1 shows the bandwidth in consideration of brightness components and color difference component (Cb, Cr). The last multiplier 2 shows writing and reading to and from the external memory. The first term of Expression 2 shows writing of a predicted image. The second term shows reading of a search range (predicted image) (brightness components only). The third term shows reading of color difference components of the predicted image. The multipliers 2 in the second and third terms of Expression 2 are caused by bidirectional inter-frame prediction, which is described later. In the prior art, all of these are realized by the transfer to/from the external memory which is connected with one data bus.
For example, as described in IEEE JOURNAL OF SOLID-STATE CIRCUITS, VOL.32, NO.11, NOVEMBER 1997, when 32-bit width data is transferred in the 81-MHz operation, the bandwidth is 2.6 Gbp. In this case, there is an allowance about 1.5 times as large as required data transfer amount (1.7 Gbp). They are practical transfer rate and data width at this stage, considering codes at the output stage being temporarily buffered in the external memory, overhead of data transfer, and LSI cost depending on the number of pins greatly.
As described above, the prior art image coding apparatus stores plural pieces of frame data such as a predicted image and search image in the outboard memory, and executes the image coding by the data transfer to/from the outboard memory. However, when the search range is extended to obtain a higher image quality or the throughput is increased like HD (high density) images (for example, 1920 pixelsxc3x971152 linesxc3x9730 frames/sec), the bandwidth required for transferring data from the frame memory to a coding unit is dramatically increased. In this case, high-speed transfer or high parallel input is required to construct the image coding apparatus, whereby the system cost is increased due to an increased number of data pins.
Especially, in the motion vector detection process which is required for the image coding using the motion compensation inter-frame prediction, the detection of a block with high coding efficiency by performing pattern matching is required. Further, screens will be extended in the future and demands for higher definition and higher image quality will be growing accordingly. In order to meet these demands, the detection in a larger search range will be inevitably needed. Accordingly, the data transfer amount is significantly increased.
It is an object of the present invention to provide an image coding apparatus and its motion vector detection method, which can reduce the system cost also when the throughput of image data is increased.
Other objects and advantages of the present invention will become apparent from the detailed description and specific embodiments described are provided only for illustration since various additions and modifications within the spirit and scope of the invention will be apparent to those of skill in the art from the detailed description.
An image coding apparatus according to a 1st aspect of the present invention comprises: an image input unit; a coding unit including a motion compensation unit for performing the motion compensation inter-frame prediction, a code generation unit for coding an output result of the motion compensation unit, and an image reproduction unit for reconstructing an image on the basis of a code generation result; a code output unit for outputting a code generated by the coding unit; a search range frame memory for containing the output of the image reproduction unit; a coding target frame memory for containing frame data which is output by the image input unit; a first data bus for performing data transfer from the search range frame memory to the coding unit; and a second data bus for performing data transfer from the coding target frame memory to the coding unit, and the motion compensation unit selects a block in a search range, using the search range which is input from the search range frame memory via the first data bus and a coding target block which is input from the coding target frame memory via the second data bus, and performs the motion compensation. Therefore, the transfer amount is dispersed and the frame memories are selected according to the transfer amount or memory capacity, whereby an effective construction can be realized. Accordingly, even when the throughput of image data is increased, a rise in the transfer frequency or an increase in the number of data pins is suppressed, whereby an image coding apparatus having a lower system cost can be provided.
According to a 2nd aspect of the present invention, in the image coding apparatus of the 1st aspect, the motion compensation unit and the search range frame memory are integrated on one chip. Therefore, the data amount transferred from the coding target frame memory as an outboard memory is reduced, whereby an image coding apparatus having a lower system cost can be provided.
According to a 3rd aspect of the present invention, in the image coding apparatus of the 1st aspect, a memory capacity of the coding target frame memory is larger than a memory capacity of the search range frame memory. Therefore, an image coding apparatus having a lower system cost can be provided by increasing a less expensive memory which does not require a higher-speed operation with relative to the search range frame memory.
An image coding apparatus according to a 4th aspect of the present invention comprises: an image input unit; a spare vector detection unit; a coding unit including a motion compensation unit for performing the motion compensation inter-frame prediction, a code generation unit for coding an output result of the motion compensation unit, and an image reproduction unit for reconstructing an image on the basis of a code generation result; a code output unit for outputting a code generated by the coding unit; a search range frame memory for containing the output of the image reproduction unit; a coding target frame memory for containing frame data output by the image input unit; a first data bus for performing data transfer from the search range frame memory to the spare vector detection unit and the coding unit; and a second data bus for performing data transfer from the coding target frame memory to the coding unit, and the spare vector detection unit selects a block in a search range, using the search range which is input from the search range frame memory via the first data bus and a coding target block which is input by the image input unit, and calculates a motion vector, and the motion compensation unit selects a block in a search range, using the search range which is input from the search range frame memory via the first data bus on the basis of the motion vector calculated by the spare vector detection unit and the coding target block which is input from the coding target frame memory via the second data bus, and performs the motion compensation. Therefore, data of the search range from the search range frame memory, which has a large transfer amount and is used in the motion compensation unit can be reduced, whereby the transfer amount from the search range frame memory can be reduced. Accordingly, a rise in the transfer frequency or an increase in the number of data pins is suppressed, whereby an image coding apparatus having a lower system cost can be provided. Besides, the coding target block is input to the spare vector detection unit directly from the image input unit, whereby addition of data buses or an increase in the transfer amount is suppressed and an increase in the number of data pins is suppressed. Accordingly, a rise in the cost can be suppressed.
An image coding apparatus according to a 5th aspect of the present invention comprises: an image input unit; a vector estimation unit; a coding unit including a motion compensation unit for performing the motion compensation inter-frame prediction, a code generation unit for coding an output result of the motion compensation unit, and an image reproduction unit for reconstructing an image on the basis of a code generation result; a code output unit for outputting a code generated by the coding unit; a search range frame memory for containing the output of the image reproduction unit; a coding target frame memory for containing frame data output by the image input unit; a first data bus for performing data transfer from the search range frame memory to the coding unit; and a second data bus for performing data transfer from the coding target frame memory to the coding unit, and the vector estimation unit calculates a motion vector from continuous first and second frame images which are successively input by the image input unit, and the motion compensation unit selects a block in a search range, using the search range which is input from the search range frame memory via the first data bus on the basis of the motion vector calculated by the vector estimation unit and a coding target block of the second frame image which is input from the coding target frame memory via the second data bus, and performs the motion compensation. Therefore, data of the search range from the search range frame memory, which has a large transfer amount and is used in the motion compensation unit can be reduced, whereby the transfer amount from the search range frame memory can be reduced. Accordingly, a rise in the transfer frequency or an increase in the number of data pins is suppressed and an image coding apparatus having a lower system cost can be provided. Besides, the frame image is input to the vector estimation unit directly from an image input unit, whereby addition of data buses or an increase in the transfer amount is suppressed and an increase in the number of data pins is suppressed. Therefore, a rise in the cost can be suppressed.
A motion vector detection method according to a 6th aspect of the present invention comprises: performing first motion vector detection using an input coding target block and a search range which is set for the coding target block in a frame image obtained according to the motion compensation inter-frame prediction, with predetermined search precision; and performing second motion vector detection using the coding target block which is temporarily stored after being input and a search range which is set in the frame image obtained according to the motion compensation inter-frame prediction on the basis of a motion vector detected by the first motion vector detection, with higher search precision than the predetermined search precision. Therefore, data of the search range which is used in the second motion vector detection can be reduced. Besides, the coding target block which is input directly is utilized in the first motion vector detection, whereby addition of data buses or an increase in the transfer amount is suppressed.
A motion vector detection method according to a 7th aspect of the present invention comprises: estimating a motion vector from continuous first and second frame images which are successively input; and detecting a motion vector using a coding target block in the second frame image, which block is temporarily stored after being input, and a search range which is set in a frame image obtained according to the motion compensation inter-frame prediction on the basis of the motion vector detected by the estimation of the motion vector. Therefore, data of the search range which is used in the motion vector detection can be reduced. Besides, the coding target block which is directly input is utilized in the motion vector estimation, whereby the addition of data buses or an increase in the transfer amount can be suppressed.